Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

[logstash] add security example #392

Merged
merged 6 commits into from
Jun 18, 2020
Merged

Conversation

jmlrt
Copy link
Member

@jmlrt jmlrt commented Dec 2, 2019

This PR add a security example to use Logstash helm chart and Elasticsearch helm chart with TLS encryption.

Fix #587

  • Chart version not bumped (the versions are all bumped and released at the same time)
  • README.md updated with any new values or changes
  • Updated template tests in ${CHART}/tests/*.py
  • Updated integration tests in ${CHART}/examples/*/test/goss.yaml

@rhizoet
Copy link

rhizoet commented Apr 30, 2020

Any news to this?

I've used the example for my logstash setup. But I get an Error and because of this logstash become not ready:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2020-04-30T10:19:27,135][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-04-30T10:19:29,214][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"signed fields invalid"}
[2020-04-30T10:19:29,299][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2020-04-30T10:19:31,000][INFO ][org.reflections.Reflections] Reflections took 82 ms to scan 1 urls, producing 20 keys and 40 values 
[2020-04-30T10:19:31,924][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been created for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-04-30T10:19:31,928][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x793149aa run>"}
[2020-04-30T10:19:33,529][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-04-30T10:19:33,603][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-04-30T10:19:33,797][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-04-30T10:19:33,807][INFO ][org.logstash.beats.Server][main] Starting server on port: 5044
[2020-04-30T10:19:34,317][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-04-30T10:19:59,298][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"signed fields invalid"}

I've created the certs and credentials as stated in the Makefile from Elasticsearch. So nothing else done. Config is the same as in this PR.

Any idea what went wrong?

@jmlrt jmlrt marked this pull request as ready for review June 18, 2020 08:28
@jmlrt jmlrt requested a review from a team June 18, 2020 08:28
fatmcgav
fatmcgav previously approved these changes Jun 18, 2020
Copy link
Contributor

@fatmcgav fatmcgav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@fatmcgav fatmcgav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jmlrt jmlrt merged commit 7f101f2 into elastic:master Jun 18, 2020
@jmlrt jmlrt deleted the logstash-security branch June 18, 2020 12:33
@jmlrt
Copy link
Member Author

jmlrt commented Jun 18, 2020

@rhizoet The PR is merged, FYI logstash-output-elasticsearch doesn't seem to accept .pem certificate. I updated the Elasticsearch Makefile to use .crt certificate instead.

jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
@jmlrt
Copy link
Member Author

jmlrt commented Jun 18, 2020

backported to 6.8, 7.8and 7.x branches

jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
@jmlrt jmlrt mentioned this pull request Jun 18, 2020
@jmlrt jmlrt mentioned this pull request Oct 28, 2020
This was referenced Nov 17, 2020
This was referenced Nov 19, 2020
@jmlrt jmlrt mentioned this pull request Feb 8, 2021
This was referenced Mar 15, 2021
@jmlrt jmlrt mentioned this pull request May 25, 2021
@jmlrt jmlrt mentioned this pull request Mar 8, 2022
@jmlrt jmlrt mentioned this pull request Apr 21, 2022
This was referenced Sep 14, 2022
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[logstash] mounting PEM certificate to connect to Elasticsearch via TLS isn't working
3 participants